COMMENT ⊗   VALID 00002 PAGES
C REC  PAGE   DESCRIPTION
C00001 00001
C00002 00002	∂16-APR-76  0635	FTP:Joe Weizenbaum (JOSEPH @ MIT-MC) at MIT-MC    
C00024 ENDMK
C⊗;
∂16-APR-76  0635	FTP:Joe Weizenbaum (JOSEPH @ MIT-MC) at MIT-MC    
Date: 16 APR 1976 0926-EST
From: Joe Weizenbaum (JOSEPH @ MIT-MC) at MIT-MC
Sent-by: GLS at MIT-MC
To: jmc at SU-AI









                          A RESPONSE TO JOHN McCARTHY
                                      by
                               JOSEPH WEIZENBAUM

           Whatever the merit  of John McCarthy's  review of Cλ←oλ←mλ←pλ←uλ←tλ←eλ←rλ← λ←←Pλ←oλ←wλ←eλ←rλ←
   aλ←nλ←dλ← λ←←Hλ←uλ←mλ←aλ←nλ← λ←Rλ←eλ←aλ←sλ←oλ←nλ← may be, it is  all but undone by his repeated assertion
   that the  positions taken  in the  book  are derived  from a  "new  left"
   political  ideology.   Not long ago the  terms "pinko" or "commie" served
   the function McCarthy assigns to "new left" in his review.   I would have
   thought,  but for this exhibit, that even  people with as limited a sense
   of history as  John McCarthy displays  in his review  might have  learned
   something  from the  events of the  tragic decades the  United States has
   just passed through.   I would have  thought, but for this exhibit,  that
   all  participants in scholarly  debates had by  now renounced argument by
   irrelevant political association.

           McCarthy's warning to  "the outside  observer" that  the book  is
   motivated  by a  struggle over  academic appointments,  tenure decisions,
   etc.   going on within M.I.T. is bizarre and absurd.   It is another blot
   on  the review and, I would say, on its author.   Such tactics are simply
   indecent.

           I am disturbed by John  McCarthy's misreading of my book's  "main
   points."   The  books actual main point, to  the extent that there is one
   main point, is that no single way of seeing the world, whether it be that
   of the  computer metaphor,  of science,  of religion,  of some  political
   dogma,  or of  whatever, is sufficient  to yield an  understanding of the
   world worthy of the human potential to understand.

           McCarthy sees the book making the following main points:


   1) COMPUTERS CANNOT BE MADE TO REASON AS POWERFULLY AS HUMANS

           I wrote:  "...I  see no  way to  put  a bound  on the  degree  of
   intelligence  [a computer]  could, at least  in principle, attain."   (p.
   210) I then go on to argue that a computer's socialization, that is,  its
   aquisition  of  knowledge  from  its  experience  with  the  world,  must
   necessarily be  different from  the  socialization of  human  beings.   A
   computer's  intelligence  must  therefore   be  always  aλ←lλ←iλ←eλ←nλ←  to   human
   intelligence with respect to a certain range of human affairs.  (p.  213)
   Nowhere do I limit the computer's "reasoning power."   The whole book is,
   however, an attack on  the dogmatic coupling  of reason to  power.   This
   coupling  is so much part of the Zeitgeist that single-mindedly committed
   technological enthusiasts  simply  cannot  conceive of  a  discussion  of
   reason  -- whether by  computers or not --  that is not  at the same time
   centered on questions of power.   McCarthy's gratuitous projection of his
   own  preoccupations unto me in the form  of his attribution to me of this
   "main point" is further evidence for that.

!

                                                                      PAGE 2





   2) THERE ARE TASKS THAT COMPUTERS SHOULD NOT BE PROGRAMMED TO DO.


           Yes, that is genuinely a main point of the book.  And McCarthy is
   right in observing that task that should not be done at all should not be
   done by computers either.  McCarthy and I agree that psychotherapy should
   under  some  circumstances  be  practiced.    I  am  opposed  to  machine
   administered psychotherapy and McCarthy cannot see what objections  there
   might  be to it (other than those that arise from "new left" motivations)
   if it were to "cure" people.   Prefontal lobotomy "cures" certain  mental
   disorders.   But  at what price to  the patient and, I  would add, to the
   surgeon as well?  I believe that machine administered psychotherapy would
   induce an image of what it means to be human that would be  prohibitively
   costly  to human culture.   One may  disagree with this belief.   But one
   would first have to understand it and to take it into account.
   

            Elsewhere I say that an individual is dehumanized whenever he is
   treated as  less than  a whole  person.   The relevance  of that  to  the
   present  discussion  can  be  seen if  one  recalls  how  inhumanely many
   surgeons treat their  patients and people  generally.   They have,  after
   many  years of  seeing their  patients mainly  as objects  to be  cut and
   sewed, come to  see them  as nothing more  than objects.   Many  surgeons
   eventually see everyone, most importantly themselves, in this narrow way.
   Similar  remarks apply to other  professions.   It is of course necessary
   for all of us to adobt an effectively clinical attitude toward people  we
   deal  with  in a  large variety  of  situations.   The surgeon  could not
   actually cut into living flesh were he not able to impose a psychological
   distance between  himself and  his patient  while actually  wielding  the
   scalpel.   But  somewhere in  his inner  being he  should hold  on to his
   perception of his patient as a whole person.   Even more importantly, the
   patient must never be led into a situation in which he is forced, or even
   merely  encouraged, to regard himself as a mere object.   My fear is that
   computer administered psychotherapy necessarily induces just this kind of
   self-image in the patients who would be subject to it.   That, basically,
   is  my objection  to it.   I  cannot see how  such a  system could "cure"
   people in  any  reasonable  sense of  the  word  "cure", that  is,  in  a
   sufficiently encompassing interpretation of that word.

   3) SCIENCE HAS LED PEOPLE TO A WRONG VIEW OF THE WORLD AND LIFE.

           There is no "correct" or "wrong" view of life to which science or
   anything  else can lead.   The  point McCarthy here  misconstrues is that
   science, or any other system of thought, leads to an impoverished view of
   the world and  of life  when it or  any system  is taken to  be the  only
   legitimate perspective on the world and on life.

!

                                                                      PAGE 3





   4) SCIENCE IS NOT THE SOLE OR EVEN THE MAIN SOURCE OF KNOWLEDGE.

           How  reliable  would  McCarthy  say  is  his  knowledge  that his
   children are biologically his children or that the person he knows as his
   father  is  his  biological  father?   Is  science  his  source  of  such
   knowledge?   What  proportion of the truly important actions McCarthy has
   taken in the  course of  his adult life  were predicated  solely or  even
   mainly on knowledge he validated by appeals to science?  Did the ancients
   have  reliable knowledge?   Or have we  had reliable knowledge only since
   the founding of the British Royal Society  -- or that of the Stanford  AI
   Lab -- or not yet at all?


   5) CERTAIN PEOPLE AND INSTITUTIONS ARE BAD.

   
           I  know  of very  few people  I  would call  "bad" --  Hitler and
   Himmler are examples.   I  think some  of the people  McCarthy lists  are
   often wrong and sometimes behave irresponsibly especially when they speak
   to  and write for lay audiences.   There  appears to be wide agreement on
   that within the AI community itself.  I think the views expressed by some
   of the people mentioned are dangerous.   These views should be discussed,
   not  suppressed.   My book contributes to the required discussion.   Some
   people will surely find my  views wrong and perhaps even  dangerous.   If
   they  think my  views worthy of  more than contempt,  they should discuss
   them.   Does McCarthy think I am "bad?"   I don't believe so.   Why  then
   should he believe I think the people he mentions are bad?

   
           The  Department of  "Defense" is, in  my view, on  the whole bad.
   And the quotation marks are, I  would guess, entirely appropriate in  the
   eyes of most of the people of the world -- especially in the eyes of many
   who have read Orwell.   If the emperor wears no clothes, we should say he
   wears no clothes.

   Other remarks:
   

           I do not say  and I do  not believe that  "if the problem  hasn't
   been  solved in twenty years, we should give up."   I say (p.   l98) "...
   it would  be  wrong  ...   to make  impossibility  arguments  about  what
   computers can do entirely on the grounds of our present ignorance."  That
   is quite the opposite of what McCarthy charges me with saying.

           I  do  not say  or imply  that  "the Defense  Department supports
   speech recognition research  in order to  be able to  snoop on  telephone
   conversations."   Attributing  this view  to me is,  in McCarthy's words,
   "biased, baseless, false, and [seemingly] motivated by malice."  I wrote:
   "This project then represents, in the  eyes of its chief sponsor, a  long
   step  toward a  fully automated  battlefield."   I then  state my opinion
!

                                                                      PAGE 4





   that, should we get speech  recognition, large organizations such as  the
   government would use it for snooping, etc.  I believe that.  My belief is
   buttressed  by the  revelations (N.Y.T.,  August 3l  l975) that  the "NSA
   eavesdrops   on  virtually  all  cable,  Telex  and  other  nλ←oλ←nλ←-λ←tλ←eλ←lλ←eλ←pλ←hλ←oλ←nλ←eλ←
   communications leaving the  U.S. and  uses computers to  sort and  obtain
   intelligence  from  the  contents  ..."   (emphasis  mine).   Clearly the
   exclusion   of  telephone   communications  from  this   operation  is  a
   consequence of only technical limitations that would be removed if we had
   automatic speech recognition systems.

           The reference cited in note 9, page 286, is anonymous because the
   person in  question granted  me  permission to  quote from  his  internal
   memorandum on the condition that I not cite his name.   It is not nice of
   John McCarthy to press me to violate my word.

           I do not "idealize  the life of primitive  man."   It is a  cheap
   shot  often practiced by  technological enthusiasts to  charge anyone who
   mentions a loss  entailed by  man's commitment to  technology (and  there
   surely  have  been  and   are  losses)  with   advocating  a  return   to
   pretechnological  times.   Every modern writer I know of knows that there
   cannot have been a pretechnological time in the history of what we  would
   call man, and that history cannot be reversed.   But it is important that
   we recognize  and  understand  the  costs  associated  with  our  current
   commitment to technology and that we seek for ways to reduce the costs we
   deem too high.   There is nothing anti-technological, anti-scientific, or
   anti-intellectual in that.

           McCarthy suggests that my statement "Those who know who and  what
   they are do not need to ask what they should do" is "menacing" in that he
   believes  it to require  a priesthood to  apply it to  a particular case.
   The statement appears in a context (p.   273) in which I had just alluded
   to  the fact that  people are constantly asking  experts what they should
   do.   I don't believe people need  "expert" guidance on moral  questions.
   The  statement,  on its  very  face, argues  that  no priesthood  is ever
   necessary to tell  people what  they must do.   I  find McCarthy's  exact
   opposite analysis extremely puzzling.

           My  assertion that "An  individual is dehumanized  whenever he is
   treated as less than a whole person" is simply a statement of  fact.   It
   is incomprehensible to me that shame or guilt fall on me because McCarthy
   believes similar statements to be part of the catechism of the "encounter
   group movement," about which, by the way, I know next to nothing.

           Does  John McCarthy have  a logical calculus  within which he has
   proved that any idea held by the new left or the encounter group movement
   or by Mumford, Roszak,  or Ellul is  wrong, a menace,  and certain to  be
   used  as part of the arsenal of "priests [who] quickly crystallize around
   any  potential  center  of  power?"   (Here  again  we  see  evidence  of
   McCarthy's preoccupation with power.)

!

                                                                      PAGE 5





           Finally,  McCarthy asserts "Philosophical  and moral thinking has
   never found a  model of man  that relates human  beliefs to the  physical
   world  in a  plausible way."   Only someone  who has  mastered the entire
   philosophical and moral literature could have the authority to say  that.
   What truly God-like humility!   The distance that separates John McCarthy
   from Joseph Weizenbaum is truly measured by the challenges these two hurl
   at one  another:  McCarthy  defies  Weizenbaum  to  "Show  me  a  way  to
   knowledge  besides science!"   And Weizenbaum  responds:  "Can there be a
   way toward an authentic model of man that does not include and ultimately
   rest on philosophical and moral thinking?"

           No wonder we talk past one another.

-------

Message to Fredkin:
Weizenbaum says that he quotes your memo without attribution, because you
said he could quote it under those conditions.  As I recall, you told
me that he had simply announced to you that you would be quoted without
attribution.  What is your recollection?